On Blockchain
__TOC__ General * Blockchain ** a distributed database that maintains a continuously-growing list of records called blocks. * A Next-Generation Smart Contract and Decentralized Application Platform * How Bitcoin Works Under the Hood (July 14, 2013) * Private blockchain * On Public and Private Blockchains (August 7th, 2015) * What are the best open source projects to study blockchain technology in terms of simplicity? * What is the best programming language to learn if you want to work on the blockchain? * Bitcoin mining the hard way: the algorithms, protocols, and bytes * A 101 Noob Intro to Programming Smart Contracts on Ethereum * [https://www.linkedin.com/pulse/blockchain-database-adi-ben-ari Blockchain as a Database] (July 31, 2015) * State of the ÐApps Smart Contract * Smart Contract (on Wikipedia) RAFT * The Raft Consensus Algorithm Bitcoin * Bitcoin Developer Guide * Mastering Bitcoin (O'reilly, Andreas M. Antonopoulos) * Bitcoin Wiki * Proof of Work ** Proof of work ** The Proof-of-Work Concept (June 24, 2013) Ethereum * Ethereum Wiki * A Next-Generation Smart Contract and Decentralized Application Platform * Ethereum Design Rationale * Ethereum Development Tutorial * Ethereum Homestead Documentation ** the reference documentation accompanying the Homestead release of the Ethereum project. * Ethereum Builder's Guide * Awesome Ethereum Virtual Machine * Concepts ** Ether *** 1 wei = 10-18 ether *** 1 Gwei (shannon) = 109 wei = 10-9 ether ** Account Types, Gas, and Transactions *** Externally Owned Account, Contract Account, Transaction, Message, Gas ** What is Gas? ** What is coinbase? (Apr 16 '16) * Specs ** Ethereum Natural Specification Format ** Ethereum Chain Spec Format *** Subformat: genesis ** Enode URL format * APIs ** JSON RPC API (Generic) *** A transaction object ** JavaScript API ** Management APIs * Clients ** GETH & ETH Command line tools for the Ethereum Network ** How to call “geth attach” on --testnet better than writing full path? (Jan 9 '17) *** geth attach http://127.0.0.1:8545 * Test Network, Private Network ** Running geth *** Full node on the main Ethereum network, Full node on the Ethereum test network, Operating a private network ** Private network ** The Geth’s saga: setting up Ethereum private network on windows (Sep 25, 2017) ** Setting up private network or local cluster ** Running Ethereum testnet using Geth ** Ethereum testnets ** ethereum-private-network *** Instructions on how to setup an ethereum private network ** ADDING NODES TO YOUR PRIVATE BLOCKCHAIN ** [https://ethereum.stackexchange.com/a/2377 Genesis block Explanation] * Keys and Address ** Accounts, Addresses, Public And Private Keys, And Tokens ** Understanding the concept of Private Key, Public Key and Address in Ethereum Blockchain (2017/11/17) ** Location of public and private keys of Ethereum account (Apr 25 '17) *** The private key is never (or should never be) saved unencrypted on disk. It is generated from the info in the keystore file when the user enters the password. ** how to produce enode from node key? (Oct 21 '17) *** bootnode -nodekeyhex 59233b25bfa4c214a8713e07a395a5d11478de10f36c6c80ba5369541f73bc44 -writeaddress ** Create full Ethereum wallet, keypair and address (Feb 15, 2017) ** What does v, r, s in eth_getTransactionByHash mean? (May 3 '17) ** Get public key of any ethereum account (Mar 31 '17) * Programming ** A Gentle Introduction to Ethereum Programming, Part 1 (Nov 15, 2017) ** A Gentle Introduction to Ethereum Programming, Part 2 (Nov 17, 2017) ** A Gentle Introduction to Ethereum Programming, Part 3 (Jan 13, 2018) ** How can I get the data of the latest 10 blocks via web3.js? (Feb 25 '16) *** web3.eth.getBlock(web3.eth.blockNumber - i) * Services ** a Block Explorer and Analytics Platform for Ethereum * Recommendation ** 5 Essential Ethereum Dapp Tools ** 11 Best Ethereum Development Tools to Grow Your Stack (Feb 25, 2017) * misc ** Example genesis.json Go Ethereum * https://github.com/ethereum/go-ethereum * Desc. : Official golang implementation of the Ethereum protocol * License * References ** Go Ethereum Wiki ** geth Command Line Options ** JavaScript Console *** $ geth attach ipc:/some/custom/path, $ geth attach http://191.168.1.1:8545, $ geth attach ws://191.168.1.1:8546 ** JSON RPC API (web3, net, eth, db, ssh) ** JavaScript API ** Management APIs (admin, debug, miner, personal, txpool) * Readings ** Executing custom JavaScript Snippets through Non-Interactive console (geth --exec) (Jun 24 '16) *** geth --exec "personal.unlockAccount('$addr', '$passwd', 0)" --verbosity 4 attach http://127.0.0.1:$rpc_port Ethereumj * https://github.com/ethereum/ethereumj * Desc. : a pure-Java implementation of the Ethereum protocol * License : LGPL v3.0 Parity * https://www.parity.io/ * Desc. : an Ethereum client, written from the ground-up for correctness-verifiability, modularisation, low-footprint and high-performance * License : GPL v3.0 * Written in : Rust * Sources : https://github.com/paritytech/parity * Readings ** Parity official documentation Solidity * https://github.com/ethereum/solidity * Desc. : a contract-oriented, high-level language for implementing smart contracts * License : GPL v3.0 * Sources : https://github.com/ethereum/solidity/ References * Solidity (on Wikipedia) ** an object-oriented programming language for writing smart contracts. * Solidity official documentation ** Data types ** Units and Globally Available Variables ** Address ** Functions * Change Log * Solidity Tutorials * Solidity Tutorials * Learn Solidity in Y minutes * Ethereum Contract ABI * Common Patterns Readings * Solidity browser based compiler * Javascript bindings for the solidity compiler ** compile function * What are *C*, *E* and *S* properties in message call return object? (Aug 1 '16) : BigNumber Examples * Voting * Ethereum Pet Shop solc-js * JavaScript bindings for the Solidity compiler * Desc. : https://github.com/ethereum/solc-js * License : MIT License web3j * https://github.com/web3j/web3j * Desc. : a lightweight, highly modular, reactive, type safe Java and Android library for working with Smart Contracts and integrating with clients (nodes) on the Ethereum network * License : Apache License 2.0 * Written in : Java * Readings ** web3j official documentation Truffle * http://truffleframework.com/ * Desc. : a development environment, testing framework and asset pipeline for Ethereum, aiming to make life as an Ethereum developer easier * License : MIT * Sources : https://github.com/trufflesuite/truffle * References ** Truffle official documentation ** Command Reference ** [http://truffleframework.com/docs/getting_started/contracts Interacting with your contracts] ** [https://github.com/trufflesuite/truffle-contract#api Truffle Contract API] ** Deployer API ** Truffle Tutorials * Readings ** The Complete Ethereum Blockchain development Tutorials (Draft) ** Building with Webpack ** “SyntaxError: Unexpected token import” on truffle test (Jul 14 '17) : require Babel ** Ethereum: Deploy on a Private Blockchain - truffle, geth, and private blockchains (07 Mar 2017) ** truffle compile error on windows (29 Jun 2017) : truffle.cmd compile * Examples ** example-truffle-library : Example project showing how to use Truffle to create javascript libraries ** truffle-webpack-demo : A demo Webpack + React App using truffle-solidity-loader ** truffle-typescript-example : Minimal example integration of Truffle + TypeScript + TestRPC run in docker, for writing smart contracts on Ethereum blockchain. Zeppelin Solidity * https://github.com/OpenZeppelin/zeppelin-solidity * Desc. : a library for writing secure Smart Contracts on Ethereum. * License : MIT * Readings ** Zeppelin-Solidity documentation ** Zeppelin API Embark * https://github.com/iurimatias/embark-framework * Desc. : Framework for serverless Decentralized Applications using Ethereum, IPFS and other platforms * License : MIT Mist Browser * https://github.com/ethereum/mist * Desc. : the tool of choice to browse and use Ðapps Ganache CLI * https://github.com/trufflesuite/ganache-cli * Desc. : simulate full client behavior and make developing Ethereum applications faster, easier, and safer. * License : MIT Remix IDE * https://github.com/ethereum/remix-ide * Desc. : a browser-based compiler and IDE that enables users to build Ethereum contracts with Solidity language and to debug transactions * License : MIT License Quorum * https://www.jpmorgan.com/global/Quorum * Desc. : an Ethereum-based distributed ledger protocol with transaction/contract privacy and new consensus mechanisms * License : LGPL v3.0 + GPL v3.0 * Written in : * Sources : ** https://github.com/jpmorganchase/quorum ** https://github.com/jpmorganchase/constellation * References ** Quorum wiki *** Running Quorum ** Provisioning *** Getting Set Up *** Running Quorum *** Installing and running Constellation *** Constellation Configuration Sample *** Quorum Examples * Readings ** BUILDING DAPPS FOR QUORUM: PRIVATE ENTERPRISE BLOCKCHAINS ** Raft-based consensus for Ethereum/Quorum ** Gas price issue in Quorum Hyperledger * Hyperledger Projects ** Hyperledger Cello ** Hyperledger Fabric ** Hyperledger Iroha ** Hyperledger Sawtooth Lake Hyperledger Fabric * https://github.com/hyperledger/fabric * Desc. : an implementation of blockchain technology, leveraging familiar and proven technologies. * License : Apache License Version 2.0 * Written in : Go * Docker repository : https://hub.docker.com/u/hyperledger/ * Docker images : https://github.com/hyperledger/fabric/tree/master/images * Projects ** hyperledger/fabric ** hyperledger/fabric-docs ** hyperledger/fabric-samples ** hyperledger/fabric-chaincode-java ** hyperledger/fabric-chaincode-node ** hyperledger/fabric-chaintool ** hyperledger/fabric-sdk-rest ** hyperledger/fabric-sdk-node ** hyperledger/fabric-sdk-java ** hyperledger/fabric-sdk-py ** hyperledger/fabric-sdk-go Readings * References ** Official documentation ** Transaction Flow ** Read-Write set semantics ** Bringing up a Kafka-based Ordering Service ** CLI ** Specification ** REST API * Concepts ** Channels *** Channel = Organizations + Ordering Service + Anchor Peers + Chaincodes + Ledger *** At least one anchor peer per organization * Setup ** Getting Setup ** Setting Up a Network ** Multichannel Setup ** Install and Instantiate ** v1.0.0-alpha2 *** sample configtx.yaml *** sample core.yaml *** sample orderer.yaml *** Getting Started ** v1.0.0-beta *** sample crypto-config.yaml * Ordering ** A Kafka-based Ordering Service for Fabric * Ledger ** Hyperledger Fabric V1.0: Block Structure (Part 1) * Chaincode ** What is chaincode? ** Chaincode API ** Java chaincode ** [http://hyperledger-fabric.readthedocs.io/en/latest/Setup/Chaincode-setup.html Writing, Building, and Running Chaincode in a Development Environment] ** Chaincode examples * SDK ** Hyperledger Fabric SDK Design Specification v1.0 * misc ** Hyperledger Fabric Approaches v1.0 with Better Scalability and Security ** Logging Control ** Hyperledger Fabric V1.0: Block Structure (Part 1) Examples * CLI based End-to-End Flow ** 1 Orderer, 3 Peer, 1 CLI ** configtx.yaml ** docker-compose.yaml ** docker-compose-no-tls.yaml ** script.sh : called by yaml file * Fabric official samples ** Build Your First Network (BYFN) * github.com/ksachdeva/hyperledger-fabric-example ** ksachdeva.github.io Demos * ITPeople Blockchain - Art Auction Demo * IBM Blockchain - Marbles Demo Sources * fabric/core/ledger/util/couchdb/couchdb.go ** 1.0.0-alpha *** handleRequest(method, connectURL string, data io.Reader, rev string, multipartBoundary string) (*http.Response, *DBReturn, error) ** 1.0.0-beta *** handleRequest(method, connectURL string, data [byte, rev string, multipartBoundary string, maxRetries int, keepConnectionOpen bool) (*http.Response, *DBReturn, error)] * fabric/core/chaincode/shim/interfaces.go ** 1.0.0-alpha *** PutState(key string, value [byte) error] *** GetState(key string) ([byte, error)] * fabric/core/ledger/ledgerconfig/ledger_config.go ** 1.0.1 Issues * FAB-6092 : Gossip error: PKIID wasn't found (10/Sep/17) * FAB-6421 : Performance and scale improvements for 1.1 (03/Oct/17) Docker Images fabric-peer Config which is printed by "docker inspect --format=' ' ..." is "Config": { "Hostname": "e7eddde82bec", "Domainname": "", "User": "", "AttachStdin": false, "AttachStdout": false, "AttachStderr": false, "Tty": false, "OpenStdin": false, "StdinOnce": false, "Env": [ "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "PEER_CFG_PATH=/etc/hyperledger/fabric", "CORE_PEER_MSPCONFIGPATH=/etc/hyperledger/fabric/msp/sampleconfig" ], "Cmd": [ "peer", "node", "start" ], "ArgsEscaped": true, "Image": "sha256:793719e9dd193f580f32c5984ac47a8c0f986819e4795c039703b26bb6ad15ce", "Volumes": null, "WorkingDir": "", "Entrypoint": null, "OnBuild": [], "Labels": { "org.hyperledger.fabric.base.version": "0.3.0", "org.hyperledger.fabric.version": "1.0.0-alpha" } }, Fabric CA * Fabric CA User’s Guide * Fabric CA Server REST API Fabric SDK for Node.js * Sources : ** https://github.com/hyperledger/fabric-sdk-node ** https://github.com/hyperledger/fabric-sdk-node/tree/master/fabric-client ** https://github.com/hyperledger/fabric-sdk-node/tree/master/fabric-ca-client * Features ** create channels ** ask peer nodes to join the channel ** install chaincodes in peers ** instantiate chaincodes in a channel ** invoke transactions by calling the chaincode ** query the ledger for transactions or blocks * Readings ** Official documentation ** [https://fabric-sdk-node.github.io/ Application developer documentations] * Examples ** Balance transfer example *** https://github.com/hyperledger/fabric-sdk-node/tree/master/examples/balance-transfer Network Setup Setup and launch Fabric CA server # Install Go 1.7+ # Set GOPATH # Install libtool and libtdhl-dev #* # sudo apt install libtool libltdl-dev # Install fabric-ca-server and fabric-ca-client Performance Tuning * JBOD or RAID * LVM (Logical Volume Manager) * Filesystem Journaling ** Disabling Journaling in Ubuntu 14.04? * ulimit, open files ** Set ulimit parameters on ubuntu (2009/06/04) ** Raising the Maximum Number of File Descriptors (Open Files) on Ubuntu 14.04 Trusty (2015/05/18) * noatime ** Reducing Disk IO By Mounting Partitions With noatime ** Is it worth to tune Ext4 with noatime? (Aug 12 '10) ** RHEL 6 Filesystem Mount Options * vm.swappiness ** Swappiness ** RHEL 6 Tuning Virtual Memory *** A low swappiness value is recommended for database workloads. ** Swap Space and Kernel Swappiness ** How do I configure swappiness? (Ask Ubuntu) * Disk I/O Scheduler ** Noop for SSD ** Linux I/O Scheduler Comparison On The Linux 3.4 Desktop(11 May 2012) ** Selecting a Linux I/O Scheduler ** Linux Change The I/O Scheduler For A Hard Disk ** SSD benchmark of I/O schedulers (April 28th, 2010) * Docker OverlayFS storage driver ** Use the OverlayFS storage driver * CouchDB ** Linux tuning for better CouchDB performance * misc ** Tuning Linux for Database performance Companions PTE * https://github.com/dongmingh/v1performance * Desc. : Performance Traffic Engine Eris * https://monax.io/ * Desc. : an open platform for developers and devops to build, ship, and run blockchain-based applications for business ecosystems. * License : https://monax.io/library/#licensing * Sources : https://github.com/eris-ltd * References ** Eris official documentation ** Eris Getting Started *** Building Your First Blockchain App with Eris (August 24, 2016) ** Eris Command Line Interface ** [https://monax.io/docs/documentation/db/latest/specifications/api/ Eris DB Web APIs] ** Jobs Specification ** Variables Specification * Readings ** Using Docker Machine With Eris ** On Eris and Tendermint: Application and Consensus (2016/03/02) ** Cannot deploy smart contract with Eris / Monax *** Could not perform pkg action service: Could not perform pkg action: Docker: {"mssage":"Cannot link to a non running container: /simplechain ** The Blockchain Explained to Web Developers, Part 1: The Theory (2016/04/28) ** The Blockchain Explained to Web Developers, Part 2: In Practice (2016/05/20) ** The Blockchain Explained to Web Developers, Part 3: The Truth (2016/06/14) ** Eris and Ethereum - Decentralized computing on a blockchain (May 20, 2015) Tendermint * https://tendermint.com/ * Desc. : Byzantine fault-tolerant replicated state machines in any programming language * License : ? IPFS * https://ipfs.io/ * Desc. : A peer-to-peer hypermedia protocol to make the web faster, safer, and more open